Imaging device and image sensor

ABSTRACT

An imaging device, includes: an imaging unit in which are disposed a plurality of pixels, each including a filter that is capable of changing a wavelength of light passing therethrough to a first wavelength and to a second wavelength and a light reception unit that receives light that has passed through the filter, and that captures an image via an optical system; an analysis unit that analyzes the image captured by the imaging unit; and a control unit that controls the wavelength of the light to be transmitted, by the filter based upon a result of analysis by the analysis unit.

TECHNICAL FIELD

The present invention relates to an imaging device and to an imagesensor.

BACKGROUND ART

An imaging device is per se known (refer to PTL 1) that includes, foreach pixel, a variable wavelength filter that passes a wavelength thatcan be varied. However, with prior art imaging devices, there has beenthe problem that sometimes the transmission wavelengths that are set forthe various pixels are not suitable for the photographic subject.

CITATION LIST Patent Literature

PTL 1: Japanese Laid-Open Patent Publication 2013-85028.

SUMMARY OF INVENTION

According to the 1st aspect, an imaging device, comprises: an imagingunit in which are disposed a plurality of pixels, each including afilter that is capable of changing a wavelength of light passingtherethrough to a first wavelength and to a second wavelength and alight reception unit that receives light that has passed through thefilter, and that captures an image via an optical system; an analysisunit that analyzes the image captured by the imaging unit; and a controlunit that controls the wavelength of the light to be transmitted, by thefilter based upon a result of analysis by the analysis unit.

According to the 2nd aspect, an imaging device, comprises: filters thatare capable of changing between a first state of passing light of afirst wavelength, and a second state of passing light of a secondwavelength; light reception units that receive light that has passedthrough the filters, and that output signals; and a control unit thatcontrols the filters to be in the first state or to be in the secondstate, based upon the signals outputted from the light reception units.

According to the 3rd aspect, an imaging device, comprises: filters thatare capable of changing between a first state of passing light of afirst wavelength from a subject, and a second state of passing light ofa second wavelength; light reception units that receive light that haspassed through the filters, and that output signals; and a control unitthat detects light from the subject, and controls the filters to be inthe first state or to be in the second state.

According to the 4th aspect, an imaging device, comprises: filters thatare capable of changing between a first state of passing light of afirst wavelength, and a second state of passing light of a secondwavelength; light reception units that receive light that has passedthrough the filters, and that output signals; a detection unit thatdetects spatial frequency components of an image based upon the signalsoutputted from the light reception unit; and a control unit thatcontrols the filters to the first state or to the second state, basedupon the spatial frequency components detected by the detection unit.

According to the 5th aspect, an image sensor, comprises: filters thatare capable of changing between a first state of passing light of afirst wavelength, and a second state of passing light of a secondwavelength; light reception units that receive light that has passedthrough the filters, and that output signals; and a control unit thatcontrols the filters to the first state or to the second state, basedupon the signals outputted from the detection unit.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing the structure of an imaging deviceaccording to a first embodiment;

FIG. 2 is a block diagram showing a portion of the structure of an imagesensor according to the first embodiment;

FIG. 3 is a figure for explanation of the structure of this image sensoraccording to the first embodiment;

FIG. 4 is a figure showing a color filter arrangement and spatialfrequency ranges due to variable wavelength filters of this image sensoraccording to the first embodiment;

FIG. 5 is another figure showing a color filter arrangement and spatialfrequency ranges due to variable wavelength filters of this image sensoraccording to the first embodiment;

FIG. 6 is yet another figure showing a color filter arrangement andspatial frequency ranges due to variable wavelength filters of thisimage sensor according to the first embodiment;

FIG. 7 is a flow chart showing an example of the operation of theimaging device according to the first embodiment;

FIG. 8 is a block diagram showing the structure of an imaging deviceaccording to a second embodiment;

FIG. 9 is a figure showing a color filter arrangement and spatialfrequency ranges due to variable wavelength filters of an image sensoraccording to this second embodiment;

FIG. 10 is another figure showing a color filter arrangement and spatialfrequency ranges due to variable wavelength filters of this image sensoraccording to the second embodiment;

FIG. 11 is yet another figure showing a color filter arrangement andspatial frequency ranges due to variable wavelength filters of thisimage sensor according to the second embodiment;

FIG. 12 is a figure showing a color filter arrangement and spatialfrequency ranges due to variable wavelength filters of an image sensoraccording to a third embodiment;

FIG. 13 is another figure showing a color filter arrangement and spatialfrequency ranges due to variable wavelength filters of this image sensoraccording to the third embodiment; and

FIG. 14 is a figure showing a color filter arrangement due to variablewavelength filters of this image sensor according to the thirdembodiment.

DESCRIPTION OF EMBODIMENTS First Embodiment

FIG. 1 is a block diagram showing the structure of an electronic camera(hereinafter termed the “camera 1”), which is an example of an imagingdevice (image-capturing device) according to a first embodiment. Thiscamera 1 is built to include a camera body 2 and an interchangeable lens3. The interchangeable lens 3 is detachable installed to the camera body2 via a mounting portion not shown in the figures. When theinterchangeable lens 3 is thus installed to the camera body 2, aconnection portion 202 on the camera body 2 side and a connectionportion 302 on the interchangeable lens 3 side are connected together,and communication between the camera body 2 and the interchangeable lens3 becomes possible.

In FIG. 1, light from a photographic subject is incident in the +Z axisdirection in FIG. 1. Moreover, as shown by the illustrated coordinateaxes, the direction forward from the drawing paper and orthogonal to theZ axis is taken as being the +X axis direction, and the directionorthogonal to the Z axis and to the X axis and downward is taken asbeing the +Y axis direction. In the following figures, taking thecoordinate axes of FIG. 1 as reference, coordinate axes are displayed sothat the orientation of each figure can be understood.

The interchangeable lens 3 includes an imaging optical system (i.e. animage formation optical system) 31, a lens control unit 32, and a lensmemory 33. The imaging optical system 31 includes a plurality of lensesthat include a focus adjustment lens (i.e. a focusing lens) and anaperture, and forms a subject image upon an image formation surface ofan image sensor (imaging element) 4 of the camera body 2.

On the basis of a signal outputted from the body control unit 21 of thecamera body 2, the lens control unit 32 adjusts the focal point positionof the imaging optical system 31 by shifting the focus adjustment lensforward and backward along the direction of its optical axis L1.Moreover, the lens control unit 32 controls the diameter of the apertureon the basis of a signal outputted from the body control unit 21 of thecamera body 2.

The lens memory 33 is, for example, built around a non-volatile storagemedium or the like. Information corresponding to the interchangeablelens 3 is stored in the lens memory 33 as lens information. Writing oflens information into the lens memory 33 and reading out of lensinformation from the lens memory 33 are performed by the lens controlunit 32.

The camera body 2 includes the body control unit 21, the image sensor 4,a memory 23, a display unit 24, and a control unit 25. The image sensor4 is, for example, a CMOS image sensor or a CCD image sensor. And theimage sensor 4 receives a light flux that has passed through the imagingoptical system 31, and captures a subject image. A plurality of pixelsthat include photoelectric conversion units are disposed in the imagesensor 4 along a row direction (i.e. the horizontal direction) which isa first direction, and along a column direction (i.e. the verticaldirection) which is a second direction and intersects the firstdirection. The photoelectric conversion units are, for example, builtaround photo-diodes (PDs). The image sensor 4 includes this plurality ofpixels having photoelectric conversion units, and is an imaging unit 4that captures a subject image via the imaging optical system 31. Theimage sensor 4 photoelectrically converts incident light and generatespixel signals, and outputs the pixel signals that have thus beengenerated to the body control unit 21. These pixel signals are signalsgenerated on the basis of electric charges that have beenphotoelectrically converted by the photoelectric conversion units.

The memory 23 is a recording medium such as, for example, a memory cardor the like. Image data and so on is recorded in the memory 23. Writingof data into the memory 23 and reading out of data from the memory 23are performed by the body control unit 21. The display unit 24 displaysimages based upon image data, information related to photography such asshutter speed and aperture value and so on, and menu screens and so on.And the control unit 25 includes a release button and setting switchesof various types such as a power supply switch and so on, and outputsactuation signals corresponding to actuation thereof to the body controlunit 21.

The body control unit 21 is built around a CPU, ROM, RAM and so on, andcontrols the various sections of the camera 1 on the basis of a controlprogram. Moreover, the body control unit 21 includes a photographic modesetting unit 211, a focus detection unit 212, an analysis unit 22 thatincludes a frequency characteristic detection unit 213, a filter controlunit 214, a first image data generation unit 215, and a second imagedata generation unit 216.

The photographic mode setting unit 211 sets a photographic mode on thebasis of an actuation signal outputted from the control unit 25. Forexample, when half press actuation of the release button has beendetected, then, on the basis of this actuation signal, the photographicmode setting unit 211 sets a first photographic mode in which a throughimage (a live view image) that will be described hereinafter, in otherwords image data for display, is generated and displayed upon thedisplay unit 24. It should be understood that it would be acceptable toarrange for this first photographic mode to be set on the basis of thepower supply switch going to ON. Moreover, when full press actuation ofthe release button has been detected, then, on the basis of thisactuation signal, the photographic mode setting unit 211 sets a secondphotographic mode in which image data for recording is generated, andthis image data is recorded in the memory 23. Here, the image data maybe image data for a still image or image data for a video image.

The focus detection unit 212 performs processing required for automaticfocus adjustment (AF) of the imaging optical system 31. In concreteterms, while shifting the focus adjustment lens of the imaging opticalsystem 31 by a predetermined distance each time along the direction ofthe optical axis, for each of these predetermined distances, the focusdetection unit 212 sequentially calculates evaluated contrast values ofthe subject image on the basis of the pixel signals from the imagesensor 4. And the focus detection unit 212 calculates the position ofthe focus adjustment lens that yields the peak of these evaluatedcontrast values, in other words their maximum value, as being itsfocused position. The focus detection unit 212 then outputs a signalrelating to the focused position that has thus been calculated to thelens control unit 32. And the lens control unit 32 performs focusadjustment by shifting the focus adjustment lens to this focusedposition.

The analysis unit 22 analyzes the subject image that has been formed bythe imaging optical system 31. For example, the analysis unit 22 detectsspatial frequency characteristics of the subject image by employing thefrequency characteristic detection unit 213. In concrete terms, thefrequency characteristic detection unit 213 detects spatial frequencycharacteristics of the subject image on the basis of the pixel signalsfrom the pixels outputted from the image sensor 4. Such spatialfrequency characteristics may, for example, obtained by frequencyanalysis of all of the image (data) or of part of the image, and arecharacteristics that represent the image by components for variousspatial frequencies (amplitude strength, signal strength). For example,the spatial frequency characteristics may be represented by a spatialfrequency distribution specifying a relationship between various spatialfrequencies and the components for each spatial frequency, and mayindicate a periodic pattern upon the image or the degree of repetitionof the image.

The frequency characteristic detection unit 213 calculates the highfrequency components of the spatial frequency of the subject image inthe horizontal direction (i.e. in the row direction of the pixel array)and the high frequency components of the spatial frequency of thesubject image in the vertical direction (i.e. in the column direction ofthe pixel array), and detects the spatial frequency characteristics ofthe subject image on the basis of the high frequency components thathave been calculated. And, as will be described in detail hereinafter,the frequency characteristic detection unit 213 calculates the highfrequency components of the spatial frequency for each color component(the R component, the G component, and the B component) of the subjectimage. The high frequency components of the spatial frequency of thesubject image are the components (the amplitude strength and the signalstrength), among the frequency components of the subject image, that areof relatively high frequency. For example, a high frequency component ofthe subject image is the component, among the spatial frequencycomponents of the subject image, of spatial frequency corresponding tothe pixel pitch d (i.e. to the gaps between the pixels) which will bedescribed hereinafter. It should be understood that the high frequencycomponents are not limited to the above; one of them may be a frequencycomponent corresponding to twice the gaps, i.e. to 2 d. Furthermore, thesum of a plurality of frequency components may also be a high frequencycomponent. Yet further, a value obtained by integrating the differencesbetween the pixel signals from adjacent pixels may also be employed as ahigh frequency component of the subject image.

If the high frequency component of the spatial frequency in thehorizontal direction is greater than the high frequency component of thespatial frequency in the vertical direction by a predetermined amount T1or more, in other words by a predetermined amount or more, then thefrequency characteristic detection unit 213 detects the spatialfrequency characteristic of the subject image as being a first spatialfrequency characteristic.

And if the high frequency component of the spatial frequency in thevertical direction is greater than the high frequency component of thespatial frequency in the horizontal direction by the predeterminedamount T1 or more, in other words by the predetermined amount or more,then the frequency characteristic detection unit 213 detects the spatialfrequency characteristic of the subject image as being a second spatialfrequency characteristic.

But if the high frequency component of the spatial frequency in thehorizontal direction and the high frequency component of the spatialfrequency in the vertical direction are approximately equal, in otherwords if the difference between these two high frequency components iswithin the predetermined amount T1 described above, then the frequencycharacteristic detection unit 213 detects the spatial frequencycharacteristic of the subject image as being a third spatial frequencycharacteristic.

In this manner, in this embodiment, the frequency characteristicdetection unit 213 detects to which of the first, the second, and thethird spatial frequency characteristics the spatial frequencycharacteristic of the subject image belongs.

The frequency characteristic detection unit 213 calculates the highfrequency components of the spatial frequencies by, for example,performing high speed Fourier transform (FFT) processing upon the pixelsignals from the image sensor 4. Moreover, it would also be acceptableto arrange for the frequency characteristic detection unit 213 tocalculate the high frequency components of the spatial frequencies byperforming high speed Fourier transform processing upon image datagenerated by the first image data generation unit 215 or by the secondimage data generation unit 216. Furthermore, instead of employing highspeed Fourier transform processing, the frequency characteristicdetection unit 213 may calculate the high frequency components of thespatial frequency by performing the following calculation processing.That is, the frequency characteristic detection unit 213 may calculatethe differences between the pixel signals of pixels that are disposed tobe adjacent in the row direction, and may calculate the high frequencycomponents of the spatial frequency in the horizontal direction byintegrating those differences. And, in a similar manner, the frequencycharacteristic detection unit 213 may calculate the differences betweenthe pixel signals of pixels that are disposed to be adjacent in thecolumn direction, and may calculate the high frequency components of thespatial frequency in the vertical direction by integrating thosedifferences.

The filter control unit 214 controls the pixel color arrangement of theimage sensor 4 on the basis of the result of analysis of the subjectimage by the analysis unit 22. It may be said that the filter controlunit 214 controls the position of the pixels of the image sensor 4 onthe basis of the result of analysis of the subject image. For example,the filter control unit 214 may control the pixel color arrangement ofthe image sensor 4 on the basis of the spatial frequency characteristicsof the subject image detected by the frequency characteristic detectionunit 213, thus changing the resolution (i.e. the resolving power) of theimage sensor 4. Although the details will be described hereinafter, thefilter control unit 214 changes the resolution by controlling thetransmission wavelengths of the variable wavelength filters of thepixels of the image sensor 4.

It should be understood that it would also be acceptable to arrange forthe filter control unit 214 to control the transmission wavelengths ofthe variable wavelength filters on the basis of the structure of theimage obtained by image capture of the photographic subject. The filtercontrol unit 214 may control the transmission wavelengths of thevariable wavelength filters on the basis of information related tocharacteristics of the photographic subject image, such as the textureof the image, the edges included in the image, periodic patterns in thepixel signals or in the image data, or the like. In this case, theanalysis unit 22 analyzes the pixel signals for the pixels or the imagedata, and generates information related to characteristics of thephotographic subject image. And the filter control unit 214 changes theresolution of the image sensor 4 by controlling the transmissionwavelengths of the variable wavelength filters on the basis of thisinformation related to characteristics of the photographic subject imagegenerated by the analysis unit 22.

If the first photographic mode is set by the photographic mode settingunit 211, then the first image data generation unit 215 performs imageprocessing of various types upon the pixel signals outputted from theimage sensor 4, and generates image data for display. And the displayunit 24 displays an image based upon this image data for display thathas thus been generated by the first image data generation unit 215.

And, if the second photographic mode is set by the photographic modesetting unit 211, then the second image data generation unit 216performs image processing of various types upon the pixel signalsoutputted from the image sensor 4, and generates image data forrecording. And the second image data generation unit 216 records thisimage data for recording that has thus been generated in the memory 23.The image processing performed by the first image data generation unit215 and the image processing performed by the second image datageneration unit 216 may, for example, include per se known imageprocessing such as tone conversion processing, edge enhancementprocessing, and so on. It should be understood that it would also beacceptable for the first image data generation unit 215 and the secondimage data generation unit 216 to be built in one unit as an image datageneration unit that generates either image data for display or imagedata for recording.

The structure of the image sensor 4 according to the first embodimentwill now be explained with reference to FIGS. 2 and 3. FIG. 2 is a blockdiagram showing a portion of the structure of the image sensor 4according to the first embodiment. And FIG. 3 is a figure forexplanation of the structure of this image sensor 4 according to thefirst embodiment. FIG. 3(a) is a sectional view showing an example ofthe cross-sectional structure of the image sensor 4, and FIG. 3(b) is aplan view for explanation of an example of the layout of transparentelectrodes of variable wavelength filters 72 of this image sensor 4.

As shown in FIG. 2, the image sensor 4 includes a plurality of pixels10, a vertical filter drive unit 41, a horizontal filter drive unit 42,a pixel vertical drive unit 43, a column circuit 44, a horizontal scanunit 45, an output unit 46, and a control unit 47. In this image sensor4, the pixels 10 are disposed in a two dimensional array (for examplealong a row direction and along a column direction that intersects thatrow direction). In the example shown in FIG. 2, in order to simplify theexplanation, only fifteen pixels 10 in the horizontal direction bytwelve pixels 10 in the vertical direction are shown, but actually theimage sensor 4 has, for example, several million to several hundredmillion pixels or more.

As shown in FIG. 3(a), the image sensor 4 includes a semiconductorsubstrate 100, a wiring layer 110, a support substrate 120, micro lenses71, and variable wavelength filters 72. In the example shown in FIG.3(a), the image sensor 4 is built as a back side illuminated type imagesensor. The semiconductor substrate 100 is laminated to the supportsubstrate 120 via the wiring layer 110. The semiconductor substrate 100is made of a semiconductor substrate material such as silicon or thelike, and the support substrate 120 is made from a semiconductorsubstrate material or a glass substrate material or the like. The wiringlayer 110 is a wiring layer that includes a conductor layer (a metalliclayer) and an insulation layer, and includes a plurality of wires andvias and so on. Copper, aluminum, or the like is employed for theconductor layer. And the insulation layer is made from an oxide layer ora nitride layer or the like. As described above, the light that passesthrough the imaging optical system 31 is principally incident along the+Z axis direction.

Each of the pixels 10 (i.e. the pixels 10 a through 10 c in FIG. 3)includes a micro lens 71, a variable wavelength filter 72, a lightshielding layer 74, and a photoelectric conversion unit 75. The microlens 71 condenses the incident light upon the photoelectric conversionunit 75. The light shielding layer 74 is disposed at a boundary betweenadjacent pixels, and suppresses leakage of light between adjacentpixels. And the photoelectric conversion unit 75 photoelectricallyconverts incident light and generates electric charge.

The variable wavelength filter 72 includes electrochromic layers(hereinafter termed “EC layers”) 61, 62, and 63 that are laminatedtogether in that order from the side of the micro lens 71 to the side ofthe semiconductor substrate 100, and transparent electrodes 51, 52, 53,and 54. The EC layers 61 through 63 are made by employing electrochromicmaterials such as metallic oxides or the like. And the transparentelectrodes 51 through 54 are, for example, made from ITO (indium tinoxide) or the like. Insulating films 73 are provided between the EClayer 61 and the transparent electrode 52, between the EC layer 62 andthe transparent electrode 53, and between the EC 63 layer and thetransparent electrode 54. Moreover, an electrolytic material layer (i.e.an electrolyte film) not shown in the figures is provided to thisvariable wavelength filter 72.

As clearly shown in FIG. 3(b), for each of the plurality of EC layers,the transparent electrodes 51 are disposed along the X direction, inother words along the row direction, so as to cover the one surfaces ofthe plurality of EC layers 61 that are disposed along the row direction.In the example shown in FIG. 2, since there are twelve rows in the arrayof pixels 10, accordingly twelve of the transparent electrodes 51 aredisposed side by side. And, in a similar manner to the case with thetransparent electrodes 51, the transparent electrodes 52 and thetransparent electrodes 53 are disposed along the X direction so as tocover the one surfaces of the plurality of EC layers 62 and theplurality of EC layers 63.

The transparent electrodes 54 are electrodes that are common to thethree EC layers 61, 62, and 63, and are disposed on the surfaces of theEC layers 63 on the other side. As clearly shown in FIG. 3(b), alongeach of the plurality of EC layers 63 that are disposed along the Ydirection which intersects the X direction, in other words along thecolumn direction, the common transparent electrodes 54 are disposedalong the column direction. In the example shown in FIG. 2, since thereare fifteen columns in the array of pixels 10, accordingly fifteen ofthe common transparent electrodes 54 are disposed side by side.

The transparent electrodes 51 through 53 and the common transparentelectrodes 54 are electrodes that are disposed in the form of a matrix(i.e. a mesh) with respect to the EC layers 61, 62, and 63. Thetransparent electrodes 51 through 53 are connected to the verticalfilter drive unit 41, and the common transparent electrodes 54 areconnected to the horizontal filter drive unit 42. Due to this, in thisembodiment, it is possible to perform active matrix driving, in whichdrive control of the EC layers 61, 62, and 63 is performed by employingthe above electrodes in matrix form.

Upon supply of a drive signal to the transparent electrode 51 and to thecommon transparent electrode 54, the EC layer 61 develops B (blue) colorby generating an oxidation-reduction reaction. Accordingly, upon supplyof such a drive signal, among the incident light, the EC layer 61transmits light of the wavelength region corresponding to B (blue).Moreover, upon supply of a drive signal to the transparent electrode 52and to the common transparent electrode 54, the EC layer 62 develops G(green) color by generating an oxidation-reduction reaction.Accordingly, upon supply of such a drive signal, among the incidentlight, the EC layer 62 transmits light of the wavelength regioncorresponding to G (green). Moreover, upon supply of a drive signal tothe transparent electrode 53 and to the common transparent electrode 54,the EC layer 63 develops R (red) color by generating anoxidation-reduction reaction. Accordingly, upon supply of such a drivesignal, among the incident light, the EC layer 63 transmits light of thewavelength region corresponding to R (red). And when the supply of suchdrive signals is stopped, the EC layers 61, 62, 63 keep developing thecolors described above over a certain time interval, and then, when areset signal is supplied, they go into a state of transparency (i.e. adecolored state) in which, among the incident light, light of allwavelength regions is transmitted.

As described above, each of the plurality of variable wavelength filters72 includes three filters: an EC layer 61 that develops B (blue) color,an EC layer 62 that develops G (green) color, and an EC layer 63 thatdevelops R (red) color. In the state in which no drive signal is beingsupplied to any of the three EC layers 61, 62, and 63, the transmissionwavelength region of the three EC layers is W (white). But when drivesignals are being supplied to all of the three EC layers 61, 62, and 63,then the transmission wavelength region of the three EC layers togetherbecomes BK (black). In a similar manner, when a drive signal is suppliedonly to the EC layer 61, or only to the EC layer 62, or only to the EClayer 63, then the transmission wavelength region of the three EC layerstogether becomes B (blue), or G (green), or R (red), respectively.

Due to this, by combining the transmission wavelengths of the EC layers61 through 63, it becomes possible for the variable wavelength filter 72principally to pass light in any of the W (white), BK (black), R (red),G (green), or B (blue) wavelength regions.

In the following explanation, a pixel for which the three-layer ECtransmission wavelength region of the variable wavelength filter 72 is B(blue) will be termed a “B pixel”, a pixel for which the three-layer ECtransmission wavelength region is G (green) will be termed a “G pixel”,and a pixel for which the three-layer EC transmission wavelength regionis R (red) will be termed an “R pixel”. In other words, a pixel in whichthe variable wavelength filter 72 is being controlled to be a B (blue)color filter will be termed a “B pixel”, and, in a similar manner, apixel in which the variable wavelength filter 72 is being controlled tobe a G (green) color filter will be termed a “G pixel”, and a pixel inwhich the variable wavelength filter 72 is being controlled to be a R(red) color filter will be termed an “R pixel”. In this manner, in eachof the pixels 10, the wavelength transmitted through the variablewavelength filter 72 is changed over as appropriate, and the opticalcomponent at this wavelength is photoelectrically converted. Thephotoelectric conversion unit 75 of each of the pixels 10photoelectrically converts the light that has passed through itsvariable wavelength filter 72. The photoelectric conversion unit 75 isalso a light reception unit 75 that receives the light from thephotographic subject that has passed through the variable wavelengthfilter 72, and the photoelectric conversion unit 75 (i.e. the lightreception unit 75) photoelectrically converts this incident light andgenerates a signal (i.e. generates electric charge).

In FIG. 2, the vertical filter drive unit 41 selects a plurality of rowsof the variable wavelength filters 72, in other words selects one ormore predetermined transparent electrodes among the plurality oftransparent electrodes 51 through 53, and supplies a drive signal orsignals thereto. And the horizontal filter drive unit 42 selects aplurality of columns of the variable wavelength filters 72, in otherwords selects one or more predetermined transparent electrodes among theplurality of common transparent electrodes 54, and supplies a drivesignal or signals thereto. In this manner, the EC layers develop colorsrelated both to the transparent electrodes 51 through 53 selected by thevertical filter drive unit 41 and to the common transparent electrodes54 selected by the horizontal filter drive unit 42.

As an example, in FIG. 3(b), if the horizontal filter drive unit 42selects the common transparent electrode 54 on the right edge from amongthe three common transparent electrodes 54 and supplies a drive signalthereto, and furthermore the vertical filter drive unit 41 selects thetransparent electrode 51 on the upper edge in FIG. 3(b) from among thenine transparent electrodes 51 through 53 and supplies a drive signalthereto, then the EC layer 61 positioned at the upper right cornerbecomes colored. Moreover, if the horizontal filter drive unit 42selects the same common transparent electrode 54 and supplies a drivesignal thereto, and furthermore the vertical filter drive unit 41selects the transparent electrode 52 on the upper edge in FIG. 3(b) andsupplies a drive signal thereto, then the EC layer 62 positioned at theupper right corner becomes colored. Yet further, if the horizontalfilter drive unit 42 selects the same common transparent electrode 54and supplies a drive signal thereto, and furthermore the vertical filterdrive unit 41 selects the transparent electrode 53 on the upper edge inFIG. 3(b) and supplies a drive signal thereto, then the EC layer 63positioned at the upper right corner becomes colored.

In FIG. 2, the control unit 47 controls the vertical filter drive unit41, the horizontal filter drive unit 42, the pixel vertical drive unit43, the column circuit unit 44, the horizontal scan unit 45, and theoutput unit 46 on the basis of signals from the body control unit 21 ofthe camera 1. For example, the control unit 47 may control the verticalfilter drive unit 41 and the horizontal filter drive unit 42 accordingto commands from the filter control unit 214 of the body control unit21. The control unit 47 sets (i.e. changes) the transmission wavelengthsof the variable wavelength filters 72 by controlling the signalsinputted to the variable wavelength filters 72 from the vertical filterdrive unit 41 and from the horizontal filter drive unit 42. And thepixel vertical drive unit 43 supplies control signals to the pixels 10on the basis of signals from the control unit 47, thus controlling theoperation of the pixels 10.

The column circuit unit 44 includes a plurality of analog/digitalconversion units (A/D conversion units), and converts the pixel signalsinputted from the pixels 10 via the vertical signal lines into digitalsignals and outputs these digital signals after conversion to thehorizontal scan unit 45. And the horizontal scan unit 45 outputs thepixel signals that have been outputted from the column circuit unit 44sequentially to the output unit 46. The output unit 46 includes a signalprocessing unit not shown in the figures which performs signalprocessing such as correlated double sampling and signal amountcorrection processing and so on upon the signals inputted from thehorizontal scan unit 45, and outputs the result to the body control unit21 of the camera 1. The output unit 46 includes input and outputcircuits corresponding to high speed interfaces such as LVDS and SLVSand so on, and transmits the above signals to the body control unit 21at high speed.

As described above, the camera 1 according to this embodiment detectsthe spatial frequency components of the subject image on the basis ofthe pixel signals from the pixels, outputted from the image sensor 4.Then, the camera 1 controls the variable wavelength filters 72 on thebasis of these spatial frequency components, and changes the resolutionof the image sensor 4 by changing over the pixel color arrangement. Inconcrete terms, the filter control unit 214 changes over the controlmode, and, for example, changes over between the pixel color arrangementin FIG. 4(a), and the pixel color arrangement in FIG. 5(a), and thepixel color arrangement in FIG. 6(a).

The filter control unit 214 has first, second, and third filter controlmodes. As described subsequently in detail, the filter control unit 214arranges R pixels, G pixels, and B pixels in the color arrangement ofFIG. 4(a) in the first filter control mode, arranges R pixels, G pixels,and B pixels in the color arrangement of FIG. 5(a) in the second filtercontrol mode, and arranges R pixels, G pixels, and B pixels in the colorarrangement of FIG. 6(a) in the third filter control mode. In otherwords, the filter control unit 214 controls the positions of the Rpixels, of the G pixels, and of the B pixels. Although the details willbe described hereinafter, with the pixel color arrangement shown in FIG.4(a), a good balance of resolutions between the horizontal direction andthe vertical direction is obtained. Moreover, with the pixel colorarrangement shown in FIG. 5(a), a higher resolution in the horizontaldirection is obtained; and, with the pixel color arrangement shown inFIG. 6(a), a higher resolution in the vertical direction is obtained.The filter control unit 214 selects one of the first, the second, andthe third filter control modes on the basis of the spatial frequencycomponents of the subject image, and then changes over the pixel colorarrangement so as to obtain resolutions that are appropriate for thesubject to be photographed.

For example, the filter control unit 214 executes the first filtercontrol mode when a subject image is captured in which the highfrequency components of the special frequency in each of the horizontaldirection and the vertical direction are almost equal, and in this casemay establish the pixel color arrangement of FIG. 4(a), with which agood balance is obtained between the resolution in the horizontaldirection and the resolution in the vertical direction. And, forexample, when a subject image is captured in which the high frequencycomponent of the special frequency in the horizontal direction isgreater than the high frequency component of the special frequency inthe vertical direction by at least a predetermined amount, then thefilter control unit 214 may execute the second filter control mode inwhich it establishes the pixel color arrangement of FIG. 5(a), withwhich a higher resolution is obtained in the horizontal direction.Furthermore, for example, when a subject image is captured in which thehigh frequency component of the special frequency in the verticaldirection is greater than the high frequency component of the specialfrequency in the horizontal direction by at least a predeterminedamount, then the filter control unit 214 may execute the third filtercontrol mode in which it establishes the pixel color arrangement of FIG.6(a), with which a higher resolution is obtained in the verticaldirection. In the following, the details of the color arrangements ofthe pixels for each of the control modes of the filter control unit 214will be explained in detail.

FIG. 4(a) is a figure showing the color arrangement of the R pixels, theG pixels, and the B pixels in the first filter control mode of thefilter control unit 214, and FIG. 4(b) is a figure showing the spatialfrequency range that can be resolved in the case of the colorarrangement shown in FIG. 4(a). In FIG. 4(a), as described above, the Rpixels 10 are pixels 10 in which the variable wavelength filters 72 arecontrolled to be R (red) filters. Moreover, the G pixels 10 are pixels10 in which the variable wavelength filters 72 are controlled to be G(green) filters, and the B pixels 10 are pixels 10 in which the variablewavelength filters 72 are controlled to be B (blue) filters.

In FIG. 4(a), the control unit 47 controls the variable wavelengthfilters 72 according to commands from the filter control unit 214 of thebody control unit 21, and sets up an arrangement of R pixels 10, Gpixels 10, and B pixels 10 according to a Bayer array. Pixel columns inwhich R pixels 10 and G pixels 10 are disposed alternatingly along therow direction at a pitch d, in other words with gaps d between them, andpixel columns in which G pixels 10 and B pixels 10 are disposedalternatingly along the row direction at the pitch d, in other wordswith gaps d between them, are set up alternatingly along the columndirection. In a Bayer array, the gaps (2 d) between R pixels in thehorizontal direction (i.e. the row direction) and the gaps (2 d) betweenR pixels in the vertical direction (i.e. the column direction) are thesame. In a similar manner, the gaps (2 d) between G pixels in thehorizontal direction (i.e. the row direction) and the gaps (2 d) betweenG pixels in the vertical direction (i.e. the column direction) are thesame, and the gaps (2 d) between B pixels in the horizontal direction(i.e. the row direction) and the gaps (2 d) between B pixels in thevertical direction (i.e. the column direction) are the same.

FIG. 4(b) shows the spatial frequency ranges that can be resolved in thecase of the Bayer array of FIG. 4(a). In FIG. 4(b), the vertical axis fydisplays spatial frequency in the vertical direction, and the horizontalaxis fx displays spatial frequency in the horizontal direction. In FIG.4(b), the spatial frequency ranges of the G pixels 10 that can beresolved are shown by the solid line, and the spatial frequency rangesof the R pixels 10 and of the B pixels 10 that can be resolved are shownby the dotted line. In this manner, with a Bayer array, the spatialfrequency ranges of the G pixels 10 that can be resolved are the same inthe horizontal direction and in the vertical direction, and moreover thespatial frequency ranges of the R pixels 10 and of the B pixels 10 thatcan be resolved are the same in the horizontal direction and in thevertical direction.

A Bayer array of R pixels, G pixels, and B pixels of this type as shownin FIG. 4(a) is employed during the second photographic mode in whichimage data for recording is generated and this image data is recorded inthe memory 23, when the third spatial frequency characteristic has beendetected by the frequency characteristic detection unit 213. Moreover,the Bayer array of FIG. 4(a) is employed, without any relationship tothe spatial frequency characteristics of the subject image, during thefirst photographic mode in which a through image, in other words imagedata for display, is being generated and is being displayed upon thedisplay unit 24.

FIG. 5(a) is a figure showing the arrangement of the colors of the Rpixels, the G pixels, and the B pixels in the second filter control modeof the filter control unit 214, in which layout of pixels of the samecolor along the horizontal direction is emphasized. And FIG. 5(b) showsthe spatial frequency ranges that can be resolved in the case of thecolor arrangement of FIG. 5(a).

In FIG. 5(a), pixel columns in which R pixels 10 are disposedsequentially in the row direction at the pitch d, pixel columns in whichG pixels 10 are disposed sequentially in the row direction at the pitchd, and pixel columns in which B pixels 10 are disposed sequentially inthe row direction at the pitch d are established repeatedly in turnalong the column direction. Accordingly, the R pixels 10 are disposed atintervals 3 d in the column direction, and similarly the G pixels 10 aredisposed at intervals 3 d in the column direction and the R pixels 10are disposed at intervals 3 d in the column direction. This type ofpixel color arrangement, in which layout of the R pixels, of the Gpixels, and of the B pixels along the horizontal direction isemphasized, is achieved by the control unit 47 controlling the variablewavelength filters 72 according to commands from the filter control unit214 of the body control unit 21.

Since, as described above, the R pixels are disposed at the pitch d inthe row direction and are disposed at the pitch 3 d in the columndirection, and similarly for the G pixels and for the B pixels,accordingly, as shown in FIG. 5(b), the spatial frequency ranges of theR pixels, of the G pixels, and of the B pixels that can be resolved withthis type of pixel color arrangement have resolutions in the horizontaldirection (i.e. the row direction) that are higher than the resolutionsin the vertical direction (i.e. in the column direction).

This type of arrangement shown in FIG. 5(a) of the colors of the pixels,in which layout of the R pixels, of the G pixels, and of the B pixelsalong the horizontal direction is emphasized, is employed during thesecond photographic mode, when the first spatial frequencycharacteristic has been detected by the frequency characteristicdetection unit 213.

FIG. 6(a) is a figure showing the arrangement of the colors of the Rpixels, G pixels, and B pixels in the third filter control mode of thefilter control unit 214, in which layout of pixels of the same coloralong the vertical direction is emphasized. And FIG. 6(b) shows thespatial frequency ranges that can be resolved in the case of the colorarrangement of FIG. 6(a).

In FIG. 6(a), pixel columns in which R pixels 10 are disposedsequentially in the column direction at the pitch d, pixel columns inwhich G pixels 10 are disposed sequentially in the column direction atthe pitch d, and pixel columns in which B pixels 10 are disposedsequentially in the column direction at the pitch d are set uprepeatedly in turn along the row direction. Accordingly, the R pixels 10are disposed at intervals 3 d in the row direction (i.e. along thehorizontal direction), and similarly the G pixels 10 are disposed atintervals 3 d in the row direction and the R pixels 10 are disposed atintervals 3 d in the row direction. This type of pixel colorarrangement, in which layout of the R pixels, of the G pixels, and ofthe B pixels along the vertical direction is emphasized, is achieved bythe control unit 47 controlling the variable wavelength filters 72according to commands from the filter control unit 214 of the bodycontrol unit 21.

Since, as described above, the R pixels are disposed at the pitch d inthe column direction and are disposed at the pitch 3 d in the rowdirection, and similarly for the G pixels and for the B pixels,accordingly, as shown in FIG. 6(b), the spatial frequency ranges of theR pixels, of the G pixels, and the B pixels that can be resolved withthis type of pixel color arrangement have resolutions in the verticaldirection (i.e. the column direction) that are higher than theresolutions in the horizontal direction (i.e. in the row direction).

This type of arrangement shown in FIG. 6(a) of the colors of the pixels,in which layout of the R pixels, of the G pixels, and of the B pixelsalong the vertical direction is emphasized, is employed during thesecond photographic mode, when the second spatial frequencycharacteristic has been detected by the frequency characteristicdetection unit 213.

The frequency characteristic detection unit 213 described abovecalculates the high frequency components of the spatial frequency foreach color component of the subject image. In concrete terms, on thebasis of the pixel signals from the R, G, and B pixels outputted fromthe image sensor 4, in other words on the basis of the pixel signals forthe R component, the G component, and the B component, the frequencycharacteristic detection unit 213 calculates the high frequencycomponents in the horizontal direction and the high frequency componentsin the vertical direction related to each of the R color, the G color,and the B color. In the case of the Bayer array shown in FIG. 4(a), theimage sensor 4 generates a pixel signal for the R component at everyinterval of 2 d in the horizontal direction, a pixel signal for the Gcomponent at every interval of 2 d in the horizontal direction, and apixel signal for the B component at every interval of 2 d in thehorizontal direction. Moreover, the image sensor 4 generates a pixelsignal for the R component at every interval of 2 d in the verticaldirection, a pixel signal for the G component at every interval of 2 din the vertical direction, and a pixel signal for the B component atevery interval of 2 d in the vertical direction.

The frequency characteristic detection unit 213 performs high speedFourier transform processing upon the pixel signals for the R componentat every interval of 2 d in the horizontal direction, and therebycalculates high frequency components (a signal component of spatialfrequency 1/2 d and so on) for the R component in the horizontaldirection. In a similar manner, the frequency characteristic detectionunit 213 performs high speed Fourier transform processing upon the pixelsignals for the B component at every interval of 2 d in the horizontaldirection, and thereby calculates high frequency components (a signalcomponent of spatial frequency 1/2 d and so on) for the B component inthe horizontal direction.

Furthermore, the frequency characteristic detection unit 213 performshigh speed Fourier transform processing upon the pixel signals for the Rcomponent at every interval of 2 d in the vertical direction, andthereby calculates high frequency components (a signal component ofspatial frequency 1/2 d and so on) for the R component in the verticaldirection. In a similar manner, the frequency characteristic detectionunit 213 performs high speed Fourier transform processing upon the pixelsignals for the B component at every interval of 2 d in the verticaldirection, and thereby calculates high frequency components (a signalcomponent of spatial frequency 1/2 d and so on) for the B component inthe vertical direction.

When calculating the high frequency components for the G component, thefrequency characteristic detection unit 213 generates pixel signals forthe G component at the positions of R pixels 10 or B pixels 10 that arepositioned between adjacent G pixels by performing interpolationprocessing upon the pixel signals for the G component. In other words,by performing interpolation processing, the frequency characteristicdetection unit 213 acquires pixel signals for the G component at everyinterval of d in the horizontal direction, and also acquires pixelsignals for the G component at every interval of d in the verticaldirection.

The interpolation processing for the G component in the horizontaldirection will now be explained in the following. For the pixel columnsin FIG. 4(a) in which R pixels and G pixels are disposed alternatinglyin the horizontal direction, the frequency characteristic detection unit213 interpolates the pixel signal for the G component corresponding tothe position of an R pixel by employing the pixel signals from the two Gpixels that are positioned above and below that R pixel. In a similarmanner, for the pixel columns in which G pixels and B pixels aredisposed alternatingly in the horizontal direction, the pixel signal forthe G component corresponding to the position of a B pixel isinterpolated by employing the pixel signals from the two G pixels thatare positioned above and below that B pixel.

The interpolation processing for the G component in the verticaldirection will now be explained in the following. For the pixel columnsin FIG. 4(a) in which R pixels and G pixels are disposed alternatinglyin the vertical direction, the frequency characteristic detection unit213 interpolates the pixel signal for the G component corresponding tothe position of an R pixel by employing the pixel signals from the two Gpixels that are positioned to the left and to the right of that R pixel.In a similar manner, for the pixel columns in which G pixels and Bpixels are disposed alternatingly in the vertical direction, the pixelsignal for the G component corresponding to the position of a B pixel isinterpolated by employing the pixel signals from the two G pixels thatare positioned to the left and to the right of that B pixel. It shouldbe understood that, when calculating the pixel signals for the Gcomponent in this interpolation processing, it would also be possible toperform the calculation by averaging the pixel signals from two pixels,or by calculating a weighted average of the pixel signals from twopixels.

By performing the interpolation processing described above, thefrequency characteristic detection unit 213 acquires pixel signals forthe G component at every interval of d in the horizontal direction, andalso acquires pixel signals for the G component at every interval of din the vertical direction. And the frequency characteristic detectionunit 213 performs high speed Fourier transform processing upon thesepixel signals for the G component at every interval of d in thehorizontal direction, and thereby calculates high frequency components(a signal component of spatial frequency 1/d and so on) for the Gcomponent in the horizontal direction. Furthermore, the frequencycharacteristic detection unit 213 performs high speed Fourier transformprocessing upon the pixel signals for the G component at every intervalof d in the vertical direction, and thereby calculates high frequencycomponents (a signal component of spatial frequency 1/d and so on) forthe G component in the vertical direction.

In this manner, the frequency characteristic detection unit 213calculates high frequency components for each color component in thehorizontal direction and in the vertical direction. And the frequencycharacteristic detection unit 213 performs spatial frequencycharacteristic determination processing on the basis of these highfrequency components that have thus been calculated. For example, thefrequency characteristic detection unit 213 may add together the highfrequency components in the horizontal direction related to the Rcomponent, to the G component, and to the B component to calculate theaddition high frequency component in the horizontal direction, and, in asimilar manner, may add together the high frequency components in thevertical direction related to the R component, to the G component, andto the B component to calculate the addition high frequency component inthe vertical direction. And the frequency characteristic detection unit213 may perform the spatial frequency characteristic determinationprocessing by comparing together the addition high frequency componentin the horizontal direction and the addition high frequency component inthe vertical direction. It should be understood that, instead ofcalculating the high frequency component for each of the colorcomponents, it would also be acceptable to arrange to calculate the highfrequency component by mixing together the three RGB color components.

It should be understood that it would also be acceptable to arrange forthe frequency characteristic detection unit 213 to calculate thedifferences between the pixel signals from adjacent pixels that aredisposed both in the row direction and in the column direction, tocalculate the high frequency components for the spatial frequency in thehorizontal direction and the vertical direction by integrating thosedifferences, and thereby to calculate the high frequency components foreach color component. In this case as well, when calculating the highfrequency components for the G component, the pixel signals for the Gcomponent corresponding to the positions of R pixels or of B pixelsbetween neighboring G pixels are calculated by interpolation, asdescribed above.

The filter control unit 214 selects one of the first, second, and thirdfilter control modes on the basis of the spatial frequencycharacteristics of the subject image, and changes over the pixel colorarrangement to a Bayer array, or to an array in which the layout alongthe horizontal direction is emphasized, or to an array in which thelayout along the vertical direction is emphasized.

In this manner, in this embodiment, by controlling the variablewavelength filters 72 of the image sensor 4, the camera 1 is able tocontrol the positions of the R pixels, the G pixels, and the B pixels,and thereby is able to change the resolutions in the horizontaldirection and in the vertical direction. In other words, for example,the camera 1 is able to change over between having well balancedresolutions in the horizontal direction and in the vertical direction asshown in FIG. 4, and having higher resolution in the horizontaldirection as shown in FIG. 5, and having higher resolution in thevertical direction as shown in FIG. 6.

FIG. 7 is a flow chart showing an example of the operation of thiscamera 1 according to the first embodiment. An example of the operationof the camera 1 will now be explained with reference to this flow chartof FIG. 7. The processing shown in FIG. 7 is started when, for example,the release button is half pressed by the user and the firstphotographic mode is set by the photographic mode setting unit 211.

In step S100, the filter control unit 214 executes the first filtercontrol mode and controls the variable wavelength filters 72 of thepixels 10 via the control unit 47 of the image sensor 4, and thereby, asshown in FIG. 4(a), R pixels 10, G pixels 10, and B pixels 10 aredisposed according to a Bayer array.

In step S110, the body control unit 21 causes the image sensor 4 toperform image capture. The image sensor 4 photoelectrically converts thelight from the photographic subject, and outputs a pixel signal fromeach of the pixels 10 to the body control unit 21. The first image datageneration unit 215 performs image processing upon the pixel signalsoutputted from the image sensor 4, and generates image data for display.And the display unit 24 displays a through image on the basis of thisimage data for display.

In step S120, on the basis of the pixel signals from the image sensor 4,the focus detection unit 212 sequentially calculates evaluated contrastvalues of the subject image. And the focus detection unit 212 generatesa signal related to the position of the focus adjustment lens when theevaluated contrast value reaches its maximum value, and outputs thissignal to the lens control unit 32. The lens control unit 32 then shiftsthe focus adjustment lens to its focused position, thus performing focusadjustment.

After the focus adjustment lens has been shifted to its focusedposition, in step S130, on the basis of the pixel signals outputted fromthe image sensor 4, the frequency characteristic detection unit 213calculates both the high frequency components of the subject image inthe horizontal direction and also the high frequency components of thesubject image in the vertical direction. It should be understood thatthis calculation of the high frequency components in step S130 is notnecessarily performed after the AF operation in step S120; thiscalculation could be performed before the AF operation or simultaneouslywith the AF operation, but is preferably performed after the AFoperation, in order for the calculation of the high frequency componentsof the subject image to be performed in a focused state.

In step S140, the second photographic mode is established when thephotographic mode setting unit 211 detects full press actuation of therelease button by the user.

In step S150, the frequency characteristic detection unit 213 makes adecision as to whether or not the high frequency component of thespatial frequency of the subject image in the horizontal direction isgreater than the high frequency component of the spatial frequency ofthe subject image in the vertical direction by at least thepredetermined amount T1. If the high frequency component in thehorizontal direction is greater than the high frequency component in thevertical direction by at least the predetermined amount T1, then thefrequency characteristic detection unit 213 decides that the spatialfrequency characteristic of the subject image is the first spatialfrequency characteristic and the flow of control proceeds to step S160,whereas if a negative decision is reached in step S150 then the flow ofcontrol is transferred to step S170. In step S160, on the basis ofdetection of the first spatial frequency characteristic by the frequencycharacteristic detection unit 213, the filter control unit 214 executesthe second filter control mode. In this second filter control mode, Rpixels 10, G pixels 10, and B pixels 10 are disposed as shown in FIG.5(a), and thereby the pixel color arrangement is changed over so thatthe resolution in the horizontal direction becomes higher (i.e., ischanged over to a layout in which the horizontal direction isemphasized).

In step S170, the frequency characteristic detection unit 213 makes adecision as to whether or not the high frequency component of thespatial frequency of the subject image in the vertical direction isgreater than the high frequency component of the spatial frequency ofthe subject image in the horizontal direction by at least thepredetermined amount T1. If the high frequency component in the verticaldirection is greater than the high frequency component in the horizontaldirection by at least the predetermined amount T1, then the frequencycharacteristic detection unit 213 decides that the spatial frequencycharacteristic of the subject image is the second spatial frequencycharacteristic and the flow of control proceeds to step S180. In stepS180, on the basis of detection of the second spatial frequencycharacteristic by the frequency characteristic detection unit 213, thefilter control unit 214 executes the third filter control mode. In thisthird filter control mode, R pixels 10, G pixels 10, and B pixels 10 aredisposed as shown in FIG. 6(a), and thereby the pixel color arrangementis changed over so that the resolution in the vertical direction becomeshigher (i.e., is changed over to a layout in which the verticaldirection is emphasized).

If a negative decision is reached in step S170, then the frequencycharacteristic detection unit 213 decides that the spatial frequencycharacteristic of the subject image is the third spatial frequencycharacteristic, in other words that the high frequency component in thehorizontal direction and the high frequency component in the verticaldirection are almost the same, and the flow of control is transferred tostep S190. In this step S190, on the basis of detection of the thirdspatial frequency characteristic by the frequency characteristicdetection unit 213, the filter control unit 214 executes the firstfilter control mode. According to this first filter control mode, Rpixels 10, G pixels 10, and B pixels 10 are disposed according to aBayer array, as shown in FIG. 4(a).

In step S200, the body control unit 21 causes the image sensor 4 toperform main image capture. The image sensor 4 outputs the pixel signalsof the pixels 10 that have thus been generated to the body control unit21. And the second image data generation unit 216 performs imageprocessing upon the pixel signals thus outputted from the image sensor4, and generates image data for recording. The memory 23 records thisimage data for recording. In this manner, on the basis of the spatialfrequency characteristic of the subject image as detected by thefrequency characteristic detection unit 213, the main image capture instep S200 is performed with the pixel color arrangement of FIG. 5(a), orof FIG. 6(a), or of FIG. 4(a).

According to the embodiment described above, the following beneficialoperational effects are obtained.

(1) With the imaging device 1 according to this embodiment, the filtercontrol unit 214 controls the transmission wavelengths of the variablewavelength filters 72 of the plurality of pixels 10 of the imaging unit4 (i.e. of the image sensor 4), and thus controls the positions of thepixels, on the basis of the results of analysis of the subject image bythe analysis unit 22. Due to this, it is possible to change theresolution of the image sensor 4 according to the photographic subject.

(2) With the imaging device 1 according to this embodiment, the controlunit 21 controls the transmission wavelengths of the variable wavelengthfilters 72, and thus changes over the pixel color arrangement, on thebasis of the pixel signals due to the electric charges generated by thelight reception units 75 (i.e. the photoelectric conversion units 75).Due to this, it is possible to change the resolution of the image sensor4 according to the photographic subject.

(3) With the imaging device 1 according to this embodiment, thefrequency characteristic detection unit 213 detects first spatialfrequency components and second spatial frequency components of thesubject image, and the filter control unit 214 controls the variablewavelength filters 72 on the basis of these first spatial frequencycomponents and second spatial frequency components. Due to this, it ispossible to change over the pixel color arrangement on the basis of thespatial frequency components of the subject image. As a result, it ispossible to change the resolution of the image sensor 4 according to thephotographic subject.

Second Embodiment

An imaging device according to a second embodiment will now be explainedwith reference to the drawings. With this imaging device according tothe second embodiment, along with changing the sizes of the pixel blocksthat are made up from pixels of the same color, also processing isperformed to change the pixel color arrangement on the basis of thespatial frequency characteristics of the subject image. FIG. 8 is afigure showing an example of the structure of a camera 1 which is anexample of an imaging device according to the second embodiment. Itshould be understood that, in this figure, the same reference numbersare appended to portions that are the same as or that correspond toportions of the first embodiment, and the explanation will principallyfocus upon the features of difference.

The body control unit 21 of the camera 1 includes an area change unit217 and an addition control unit 218. The area change unit 217 iscapable of changing the sizes of the pixel blocks by controlling theimage sensor 4 so that the transmission wavelength regions of thevariable wavelength filters 72 of a plurality of mutually adjacentpixels 10 become the same. In other words, the area change unit 217 iscapable of changing the sizes of the pixel blocks so that they become2×2 blocks of G pixels, or 4×4 blocks of G pixels, or 6×6 blocks of Gpixels. In a similar manner, the area change unit 217 is capable ofchanging the sizes of the pixel blocks so that they become 2×2 blocks ofR pixels or of B pixels, or 4×4 blocks of R pixels or of B pixels, or6×6 blocks of R pixels or of B pixels.

Generally, when using image data photographed by the camera 1 andrecorded in the memory 23 for reproduction upon an external displaydevice, the number of pixels displayed upon the external display deviceand the number of pixels upon the image sensor 4 are almost equal, butin some cases the number of pixels displayed upon the external displaydevice is fewer than the number of pixels upon the image sensor 4.

For example, when the number of pixels displayed upon the externaldisplay device that displays the image is almost equal to the number ofpixels upon the image sensor 4, then the area change unit 217 maydetermine the size of the pixel blocks so that each pixel block consistsof a single pixel 10. In other words, the first embodiment describedabove is an example of this case in which each pixel block consists of asingle pixel.

Furthermore, when the number of pixels displayed upon the externaldisplay device that displays the image is fewer as compared to thenumber of pixels upon the image sensor 4, then the area change unit 217may determine the size of the pixel blocks so that each pixel blockconsists of a plurality of pixels (2×2 pixels or the like).

The addition control unit 218 controls the operation of the image sensor4 on the basis of the pixel blocks determined by the area change unit217. If the pixel locks consist of a plurality of pixels, then theaddition control unit 218 causes the image sensor 4 to perform additionprocessing for adding together the signals from the plurality of pixelsthat make up each pixel block. In concrete terms, for example, as suchaddition processing, the image sensor 4 performs processing to averagethe signals from the plurality of pixels 10 while controlling switchesconnected to the floating diffusions of each of the plurality of pixels10 within the pixel block to ON and OFF. And the image sensor 4generates pixel signals by adding together the signals of the pluralityof pixels 10, and outputs them to the body control unit 21. It should beunderstood that, instead of this addition processing being performedinternally to the image sensor 4, it would also be acceptable to arrangefor the pixel signals to be outputted from the image sensor 4 and forthe addition processing to be performed externally to the image sensor4.

In the case of the first photographic mode, the first image datageneration unit 215 performs image processing of various types upon theaddition pixel signals resulting from addition of the signals from theplurality of pixels within each pixel block, and thereby generates theimage data for display. And the display unit 24 displays an image on thebasis of this image data for display generated by the first image datageneration unit 215.

However, in the case of the second photographic mode, the second imagedata generation unit 216 performs image processing of various types uponthe addition pixel signals resulting from addition of the signals fromthe plurality of pixels within each pixel block, and thereby generatesthe image data for recording. And the second image data generation unit216 causes the memory 23 to record this image data for recording thathas thus been generated.

In this embodiment, the filter control unit 214 changes over the colorarrangement of the R pixel blocks, the G pixel blocks, and the B pixelblocks determined by the area change unit 217 on the basis of the filtercontrol mode. In other words, in the first filter control mode, thefilter control unit 214 arranges the R pixel blocks, the G pixel blocks,and the B pixel blocks in the Bayer array of FIG. 9(a), as will bedescribed in detail hereinafter. In a similar manner, in the secondfilter control mode, the filter control unit 214 arranges the R pixelblocks, the G pixel blocks, and the B pixel blocks in the arrangement ofFIG. 10(a) in which the layout along the horizontal direction isemphasized, and, in the third filter control mode, the filter controlunit 214 arranges the R pixel blocks, the G pixel blocks, and the Bpixel blocks in the arrangement of FIG. 11(a) in which the layout alongthe vertical direction is emphasized. Thus, the filter control unit 214selects one of the first, second, and third filter control modes, andchanges over the pixel color arrangement so that a resolution isobtained that is appropriate for the subject of photography. In thefollowing, the details of the color arrangements of the pixels for thevarious control modes of the filter control unit 214 will be explained.

FIG. 9(a) is a figure showing the pixel color arrangement in the firstfilter control mode of the filter control unit 214, and FIG. 9(b) is afigure showing the spatial frequency ranges that can be resolved in thecase of the color arrangement shown in FIG. 9(a). In FIG. 9(a), thecontrol unit 47 of the image sensor 4 has disposed R pixel blocks 20having four R pixels 10 in a 2×2 array, G pixel blocks 20 having four Gpixels 10 in a 2×2 array, and B pixel blocks 20 having four B pixels 10in a 2×2 array according to a Bayer array.

Pixel columns in which R pixel blocks 20 and G pixel blocks 20 aredisposed alternatingly along the row direction with gaps 2 d betweenthem and pixel columns in which G pixel blocks 20 and B pixel blocks 20are disposed alternatingly along the row direction with gaps 2 d betweenthem are set up alternatingly along the column direction. In the Bayerarray, the gaps (4 d) between R pixel blocks 20 in the horizontaldirection (i.e. the row direction) and the gaps (4 d) between R pixelblocks 20 in the vertical direction (i.e. the column direction) are thesame. In a similar manner, the gaps (4 d) between G pixel blocks 20 inthe horizontal direction (i.e. the row direction) and the gaps (4 d)between G pixel blocks 20 in the vertical direction (i.e. the columndirection) are the same, and the gaps (4 d) between B pixel blocks 20 inthe horizontal direction (i.e. the row direction) and the gaps (4 d)between B pixel blocks 20 in the vertical direction (i.e. the columndirection) are the same.

FIG. 9(b) shows the spatial frequency ranges that can be resolved of theR pixel blocks 20, the G pixel blocks 20, and the B pixel blocks 20 ofthe Bayer array of FIG. 9(a). In FIG. 9(b), the spatial frequency rangesof the G pixel blocks 20 that can be resolved are shown by the solidline, and the spatial frequency ranges of the R pixel blocks 20 and ofthe B pixel blocks 20 that can be resolved are shown by the dotted line.In this manner, with a Bayer array, the spatial frequency ranges of theG pixel blocks 20 that can be resolved are the same in the horizontaldirection and in the vertical direction, and moreover the spatialfrequency ranges of the R pixel blocks 20 and of the B pixel blocks 20that can be resolved are the same in the horizontal direction and in thevertical direction.

In a similar manner to the case with the first embodiment, this type ofBayer array is employed in the first photographic mode regardless of thespatial frequency characteristics of the subject image, and is alsoemployed in the second photographic mode when the third spatialfrequency characteristic has been detected by the frequencycharacteristic detection unit 213.

FIG. 10(a) is a figure showing the pixel color arrangement in the secondfilter control mode of the filter control unit 214. And FIG. 10(b) is afigure showing the spatial frequency ranges that can be resolved in thecase of the color arrangement shown in FIG. 10(a).

In FIG. 10(a), pixel columns in which R pixel blocks 20 are disposedsequentially in the row direction with gaps 2 d between them, pixelcolumns in which G pixel blocks 20 are disposed sequentially in the rowdirection with gaps 2 d between them, and pixel columns in which B pixelblocks 20 are disposed sequentially in the row direction with gaps 2 dbetween them are established repeatedly in turn along the columndirection. Accordingly, the R pixel blocks 20, the G pixel blocks 20,and the B pixel blocks 20 are each disposed at intervals 6 d in thecolumn direction.

Since, as described above, the R pixel blocks 20, the G pixel blocks 20,and the B pixel blocks 20 are each disposed with gaps 2 d between themin the row direction and are disposed with gaps 6 d between them in thecolumn direction, accordingly, as shown in FIG. 10(b), the spatialfrequency ranges that can be resolved with this type of pixel colorarrangement have resolutions in the horizontal direction (i.e. the rowdirection) that are higher than the resolutions in the verticaldirection (i.e. in the column direction).

This type of array of R pixel blocks, G pixel blocks, and B pixel blocksillustrated in FIG. 10(a) in which the layout along the horizontaldirection is emphasized is employed when, during the second photographicmode, the first spatial frequency characteristic has been detected bythe frequency characteristic detection unit 213.

FIG. 11(a) is a figure showing the pixel color arrangement in the thirdfilter control mode of the filter control unit 214. And FIG. 11(b) is afigure showing the spatial frequency ranges that can be resolved in thecase of the color arrangement shown in FIG. 11(a).

In FIG. 11(a), pixel columns in which R pixel blocks 20 are disposedsequentially in the column direction with gaps 2 d between them, pixelcolumns in which G pixel blocks 20 are disposed sequentially in thecolumn direction with gaps 2 d between them, and pixel columns in whichB pixel blocks 20 are disposed sequentially in the column direction withgaps 2 d between them are set up repeatedly in turn along the rowdirection. Accordingly, the R pixel blocks 20, the G pixel blocks 20,and the B pixel blocks 20 are each disposed at intervals 6 d in the rowdirection.

Since, as described above, the R pixel blocks 20, the G pixel blocks 20,and the B pixel blocks 20 are disposed with gaps 2 d between them in thecolumn direction and are disposed with gaps 6 d between them in the rowdirection, accordingly, as shown in FIG. 11(b), the spatial frequencyranges that can be resolved with this type of pixel color arrangementhave resolutions in the vertical direction (i.e. the column direction)that are higher than the resolutions in the horizontal direction (i.e.in the row direction).

This type of array of R pixel blocks, G pixel blocks, and B pixel blocksillustrated in FIG. 11(a) in which the layout along the verticaldirection is emphasized is employed when, during the second photographicmode, the second spatial frequency characteristic has been detected bythe frequency characteristic detection unit 213.

Next, the operation of the camera 1 of this embodiment will beexplained. The camera 1 of this embodiment performs processing similarto that shown in the flow chart of FIG. 7. In step S100, the camera 1executes the first filter control mode, and sets the pixel colorarrangement to a Bayer array, as shown in FIG. 9(a). In step S150through step S190, the camera 1 performs processing to change the pixelcolor arrangement on the basis of the spatial frequency characteristicsof the subject image. If the first spatial frequency characteristic isdetected, then the camera 1 executes the second filter control mode andestablishes the pixel color arrangement shown in FIG. 10(a) in which thelayout along the horizontal direction is emphasized, whereas, if thesecond spatial frequency characteristic is detected, then the camera 1executes the third filter control mode and establishes the pixel colorarrangement shown in FIG. 11(a) in which the layout along the verticaldirection is emphasized. But if the camera 1 does not detect either thefirst spatial frequency characteristic or the second spatial frequencycharacteristic, then it sets the pixel color arrangement to a Bayerarray.

As described above, with this embodiment, by controlling the variablewavelength filters 72 of the image sensor 4, along with being capable ofdetermining the size of the pixel blocks, the camera 1 is also capableof varying the resolution in the horizontal direction and the resolutionin the vertical direction. In other words, for example, the camera 1 iscapable of changing over between providing a good balance of theresolutions in the horizontal direction and in the vertical direction asshown in FIG. 9, and providing a higher resolution in the horizontaldirection as shown in FIG. 10, and providing a higher resolution in thevertical direction as shown in FIG. 11.

Third Embodiment

An imaging device according to a third embodiment will now be explainedwith reference to the drawings. The main differences between this thirdembodiment and the second embodiment are as follows. In the case of thesecond embodiment, in the arrangement which emphasizes the layout alongthe horizontal direction, as shown in FIG. 10(a), in the row direction,the gaps 2 d between the R pixel blocks, the gaps 2 d between the Gpixel blocks, and the gaps 2 d between the B pixel blocks are all thesame as one another. In a similar manner, in the column direction, thegaps 6 d between the R pixel blocks, the gaps 6 d between the G pixelblocks, and the gaps 6 d between the B pixel blocks are all the same asone another. Moreover, in the case of the second embodiment, in thearrangement which emphasizes the layout along the vertical direction, asshown in FIG. 11(a), in the column direction, the gaps 2 d between the Rpixel blocks, the gaps 2 d between the G pixel blocks, and the gaps 2 dbetween the B pixel blocks are all the same as one another. In a similarmanner, in the row direction, the gaps 6 d between the R pixel blocks,the gaps 6 d between the G pixel blocks, and the gaps 6 d between the Bpixel blocks are all the same as one another.

However, in the case of this third embodiment, in the arrangement whichemphasizes the layout along the horizontal direction and in thearrangement which emphasizes the layout along the vertical direction,the mutual gaps between the G pixel blocks, the mutual gaps between theR pixel blocks, and the mutual gaps between the B pixel blocks aredifferent. The other structures are the same as those in the secondembodiment.

In this embodiment, in the first filter control mode, in a similarmanner to the case with the second embodiment, the filter control unit214 arranges the R pixel blocks 20, the G pixel blocks 20, and the Bpixel blocks 20 in the Bayer array of FIG. 9(a). And, also in a similarmanner, in the second filter control mode, the filter control unit 214arranges the R pixel blocks 20, the G pixel blocks 20, and the B pixelblocks 20 so that their layout along the horizontal direction isemphasized as in FIG. 12(a), and, in the third filter control mode,arranges them so that their layout along the vertical direction isemphasized as in FIG. 13(a).

FIG. 12(a) is a figure showing the arrangement of the colors of thepixels by the filter control unit 214 in the second filter control mode,in which the layout along the horizontal direction is emphasized. AndFIG. 12(b) shows the spatial frequency ranges that can be resolved inthe case of the color arrangement of FIG. 12(a).

In FIG. 12(a), pixel columns in which R pixel blocks 20 and B pixelblocks 20 are disposed alternatingly in the row direction with gaps 2 dbetween them and pixel columns in which G pixel blocks 20 are disposedsequentially in the row direction with gaps 2 d between them aredisposed repeatedly along the column direction. Accordingly, in the rowdirection, G pixel blocks 20 are disposed with gaps 2 d between them,and R pixel blocks 20 and B pixel blocks 20 are each disposed with gaps4 d between them. Moreover, in the column direction, G pixel blocks 20are disposed with gaps 4 d between them, and R pixel blocks 20 and Bpixel blocks 20 are each disposed with gaps 8 d between them.

In this manner, with the pixel color arrangement shown in FIG. 12(a) inwhich the layout along the horizontal direction is emphasized, the gaps4 d between the R pixel blocks 20 in the row direction become smallerthan the gaps 8 d between the R pixel blocks 20 in the column direction.In a similar manner, the gaps 2 d between the G pixel blocks 20 in therow direction become smaller than the gaps 4 d between the G pixelblocks 20 in the column direction, and also the gaps 4 d between the Bpixel blocks 20 in the row direction become smaller than the gaps 8 dbetween the B pixel blocks 20 in the column direction. Moreover, thegaps 2 d between the G pixel blocks 20 in the row direction becomesmaller than the gaps 4 d in the row direction between the R pixelblocks 20 and also smaller than the gaps 4 d in the row directionbetween the B pixel blocks 20. Furthermore, the gaps 4 d between the Gpixel blocks 20 in the column direction become smaller than the gaps 8 din the column direction between the R pixel blocks 20 and also smallerthan the gaps 8 d in the column direction between the B pixel blocks 20.

As shown in FIG. 12(b), for the spatial frequency ranges that can beresolved in the case described above of arrangement in which the layoutalong the horizontal direction is emphasized, the resolution in thehorizontal direction (i.e. the row direction) is higher than theresolution in the vertical direction (i.e. in the column direction).

This type of arrangement of the R pixel blocks, the G pixel blocks, andthe B pixel blocks in which the layout along the horizontal direction isemphasized is employed when, during the second photographic mode, thefirst spatial frequency characteristic has been detected by thefrequency characteristic detection unit 213.

FIG. 13(a) is a figure showing the arrangement of the colors of thepixels by the filter control unit 214 in the third filter control mode,in which the layout along the vertical direction is emphasized. And FIG.13(b) shows the spatial frequency ranges that can be resolved in thecase of the color arrangement of FIG. 13(a).

In FIG. 13(a), pixel columns in which G pixel blocks 20 are disposedrepeatedly in the column direction with gaps 2 d between them and pixelcolumns in which R pixel blocks 20 and B pixel blocks 20 are disposedalternatingly in the column direction with gaps 2 d between them aredisposed repeatedly along the row direction. Accordingly, in the columndirection, G pixel blocks 20 are disposed with gaps 2 d between them,and R pixel blocks 20 and B pixel blocks 20 are each disposed with gaps4 d between them. Moreover, in the row direction, G pixel blocks 20 aredisposed with gaps 4 d between them, and R pixel blocks 20 and B pixelblocks 20 are each disposed with gaps 8 d between them.

In this manner, with the pixel color arrangement shown in FIG. 13(a) inwhich the layout along the vertical direction is emphasized, the gaps 4d between the R pixel blocks 20 in the column direction become smallerthan the gaps 8 d between the R pixel blocks 20 in the row direction. Ina similar manner, the gaps 2 d between the G pixel blocks 20 in thecolumn direction become smaller than the gaps 4 d between the G pixelblocks 20 in the row direction, and also the gaps 4 d between the Bpixel blocks 20 in the column direction become smaller than the gaps 8 dbetween the B pixel blocks 20 in the row direction. Moreover, the gaps 2d between the G pixel blocks 20 in the column direction become smallerthan the gaps 4 d in the column direction between the R pixel blocks 20and also smaller than the gaps 4 d in the column direction between the Bpixel blocks 20. Furthermore, the gaps 4 d between the G pixel blocks 20in the row direction become smaller than the gaps 8 d in the rowdirection between the R pixel blocks 20 and also smaller than the gaps 8d in the row direction between the B pixel blocks 20.

As shown in FIG. 13(b), for the spatial frequency ranges that can beresolved in the case described above of arrangement in which the layoutalong the vertical direction is emphasized, the resolution in thevertical direction (i.e. the column direction) is higher than theresolution in the horizontal direction (i.e. in the row direction).

This type of arrangement of the R pixel blocks, the G pixel blocks, andthe B pixel blocks in which the layout along the vertical direction isemphasized is employed when, during the second photographic mode, thesecond spatial frequency characteristic has been detected by thefrequency characteristic detection unit 213.

Next, the operation of this embodiment will be explained. The camera 1of this embodiment performs processing similar to that shown in the flowchart of FIG. 7. In step S100, the camera 1 executes the first filtercontrol mode, and sets the pixel color arrangement to a Bayer array, asshown in FIG. 9(a). In step S150 through step S190, the camera 1performs processing to change the pixel color arrangement on the basisof the spatial frequency characteristic of the subject image. If thefirst spatial frequency characteristic is detected, then the camera 1executes the second filter control mode and establishes the pixel colorarrangement shown in FIG. 12(a) in which the layout along the horizontaldirection is emphasized, whereas, if the second spatial frequencycharacteristic is detected, then the camera 1 executes the third filtercontrol mode and establishes the pixel color arrangement shown in FIG.13(a) in which the layout along the vertical direction is emphasized.But if the camera 1 does not detect either the first spatial frequencycharacteristic or the second spatial frequency characteristic, then itsets the pixel color arrangement to a Bayer array.

As described above, in this embodiment, for example, the camera 1 iscapable of changing over between providing a good balance of theresolutions in the horizontal direction and in the vertical direction asshown in FIG. 9, and providing a higher resolution in the horizontaldirection as shown in FIG. 12, and providing a higher resolution in thevertical direction as shown in FIG. 13.

It should be understood that, when changing over to a pixel colorarrangement in which the layout along the horizontal direction isemphasized, instead of changing over to the pixel color arrangementshown in FIG. 12(a), it would also be acceptable to arrange to changeover to the pixel color arrangement shown in FIG. 14(a). In the pixelcolor arrangement shown in FIG. 14(a), pixel columns in which R pixelblocks 20 and B pixel blocks 20 are disposed alternatingly along the rowdirection with gaps 2 d between them, and pixel columns in which G pixelblocks 20 are disposed continuously along the row direction with gaps 2d between them, are disposed repeatedly in the column direction withhalf-block deviations, in other words while being shifted by intervalsof just d. The spatial frequency ranges that can be resolved in the caseof the color arrangement of FIG. 14(a) are the same as the spatialfrequency ranges shown in FIG. 12(b).

Moreover, when changing over to a pixel color arrangement in which thelayout along the vertical direction is emphasized, instead of changingover to the pixel color arrangement shown in FIG. 13(a), it would alsobe acceptable to arrange to change over to the pixel color arrangementshown in FIG. 14(b). In the pixel color arrangement shown in FIG. 14(b),pixel columns in which G pixel blocks 20 are disposed continuously alongthe column direction with gaps 2 d between them, and pixel columns inwhich R pixel blocks 20 and B pixel blocks 20 are disposed alternatinglyalong the column direction with gaps 2 d between them, are disposedrepeatedly in the column direction with half-block deviations, in otherwords while being shifted by intervals of just d. The spatial frequencyranges that can be resolved in the case of the color arrangement of FIG.14(b) are the same as the spatial frequency ranges shown in FIG. 13(b).

In this manner, when the arrangement of the color of the pixels shown inFIG. 14(a) in which the layout along the horizontal direction isemphasized and the arrangement of the color of the pixels shown in FIG.14(b) in which the layout along the vertical direction is emphasized areemployed, and also in the case of the Bayer array shown in FIG. 9, it ispossible to provide these arrangements while relatively shifting thecolor arrangement of the pixels in the odd numbered rows and the colorarrangement of the pixels in the even numbered rows by half a block eachin the column direction, in other words while shifting them by gaps(pitch) of just d.

Furthermore, the relative displacement described above, in thearrangement in which the layout along the horizontal direction isemphasized, of the color arrangement of the pixels in the odd numberedrows and the color arrangement of the pixels in the even numbered rows,the relative displacement described above, in the arrangement in whichthe layout along the vertical direction is emphasized, of the colorarrangement of the pixels in the odd numbered columns and the colorarrangement of the pixels in the even numbered columns, and a similardisplacement of a Bayer array, can each also be applied to thearrangement in which the layout along the horizontal direction isemphasized of FIG. 5(a), the arrangement in which the layout along thevertical direction is emphasized of FIG. 6(a), and the Bayer array ofFIG. 4(a).

The following variants also come within the scope of the presentinvention; and it would also be possible to combine one or more of thefollowing variant embodiments with one or more of the embodimentsdescribed above.

Variant Embodiment 1

In the first embodiment described above an example has been explained inwhich, when changing over from a Bayer array to an arrangement in whichthe layout along the horizontal direction is emphasized or to anarrangement in which the layout along the vertical direction isemphasized, the gaps in the horizontal direction and in the verticaldirection were changed for each of the R pixels 10, the G pixels 10, andthe B pixels 10. However, it would also be acceptable, while changingthe gaps in the horizontal direction and in the vertical direction asdescribed above for one type of pixels (for example, for the G pixels)among the R, G, and B pixels, not to change the gaps for the pixels ofthe other types (i.e. for the R pixels and for the B pixels), but toleave them at the same gaps in the horizontal direction and in thevertical direction. For example, changing the gaps for only the G pixelswould be effective if the frequency characteristic detection unit 213has determined that, although the high frequency components of thespatial frequencies related to the G component (i.e. the G color) aregreatly different in the horizontal direction and in the verticaldirection, the high frequency components of the spatial frequenciesrelated to the R component (i.e. for the R color) and related to the Bcomponent (i.e. for the B color) are almost equal in the horizontaldirection and in the vertical direction, or the like.

Moreover, it would also be acceptable, while changing the gaps in thehorizontal direction and in the vertical direction for the pixels of twotypes among the R, G, and B pixels (for example, for the G pixels andfor the B pixels) as described above, to keep the gaps for the remainingpixels (for example, for the R pixels) in the horizontal direction andin the vertical direction the same without changing them. For example,changing the gaps for only the G pixels and the B pixels would beeffective if the frequency characteristic detection unit 213 hasdetermined that, although the high frequency components for the spatialfrequencies related to the G component and to the B component aregreatly different in the horizontal direction and in the verticaldirection, the high frequency components of the spatial frequenciesrelated to the R component are almost equal in the horizontal directionand in the vertical direction, or the like.

The change described above of the gaps for only one type of pixels amongthe R, G, and B pixels, or the change of the gaps for two types ofpixels among the R, G, and B pixels, can also be applied to the secondembodiment or to the third embodiment. In other words, it would beacceptable to arrange to change the gaps in the horizontal direction andin the vertical direction for only pixel blocks of one type among the Rpixel blocks, the G pixel blocks, and the B pixel blocks; or,alternatively, it would be acceptable to arrange to change the gaps inthe horizontal direction and in the vertical direction for only pixelblocks of two of those types.

Variant Embodiment 2

It would also be acceptable not to control the variable wavelengthfilters 72 in the same manner over the entire image sensor 4, but ratherto control them only over the photographic subject portion so that theresolution (i.e. the resolving power) becomes appropriate. For example,if a specific photographic subject has been detected by employing a perse known photographic subject recognition technique or the like, thenonly those of the variable wavelength filters 72 corresponding to thatspecified photographic subject may be controlled.

Variant Embodiment 3

In the arrangement in which the layout along the horizontal direction isemphasized, the gaps between the pixels or between the pixel blocks arenot limited to being the gaps described above; it would also beacceptable to arrange to vary the gaps in the horizontal direction andthe gaps in the vertical direction while keeping to the condition that(the gaps in the horizontal direction)<(the gaps in the verticaldirection). In a similar manner, in the arrangement in which the layoutalong the vertical direction is emphasized, the gaps between the pixelsor between the pixel blocks are not limited to being the gaps describedabove; it would also be acceptable to arrange to vary the gaps in thehorizontal direction and the gaps in the vertical direction whilekeeping to the condition that (the gaps in the horizontaldirection)>(the gaps in the vertical direction).

Variant Embodiment 4

It would also be possible to perform analysis of the subject image (forexample, detection of its spatial frequency components) by employing asignal from a sensor other than the image sensor 4 (for example, asensor that performs processing that is required for AF, or a sensor fordetermining the amount of exposure, or the like). In this case, asubject image equivalent to the subject image formed upon the imageformation surface of the image sensor 4 by the imaging optical system 31(i.e. the image formation optical system) of FIG. 1 would be captured bythe other sensor described above, and the subject image would beanalyzed by the analysis unit 22 on the basis of the pixel signal fromthis other sensor. For example, the frequency characteristic detectionunit 213 of the analysis unit 22 may detect the spatial frequencycharacteristic of the subject image on the basis of the pixel signalfrom this other sensor. It should be understood that it is desirable forthe pixels disposed on this other sensor to be disposed at gaps equal tothose of the pixels of the image sensor 4, in order to detect highfrequency components that are similar to those in the case of theembodiments described above.

Variant Embodiment 5

In the embodiments described above, examples were explained in which thecolor arrangement of the pixels was changed over from a Bayer array toan arrangement in which the layout along the horizontal direction isemphasized or to an arrangement in which the layout along the verticaldirection is emphasized, on the basis of the result of detection by thefrequency characteristic detection unit 213. Instead of this, it wouldalso be acceptable to arrange to change over the color arrangement ofthe pixels from a Bayer array to only one of an arrangement in which thelayout along the horizontal direction is emphasized or an arrangement inwhich the layout along the vertical direction is emphasized, on thebasis of the result of detection by the frequency characteristicdetection unit 213. For example, only when changing over from a Bayerarray to an arrangement in which the layout along the horizontaldirection is emphasized, the frequency characteristic detection unit 213may calculate the high frequency component of the spatial frequency ofthe subject image in the horizontal direction, and change over may beexecuted to an arrangement in which the layout along the horizontaldirection is emphasized if this high frequency component is apredetermined value or greater. In this case of only changing over froma Bayer array to an arrangement in which the layout along the horizontaldirection is emphasized, it would also be acceptable to arrange tochange over to such an arrangement in which the layout along thehorizontal direction is emphasized, if the high frequency component inthe horizontal direction is greater than the high frequency component inthe vertical direction.

On the other hand, when only changing over from a Bayer array to anarrangement in which the layout along the vertical direction isemphasized, the frequency characteristic detection unit 213 maycalculate the high frequency component of the spatial frequency of thesubject image in the vertical direction, and change over may be executedto an arrangement in which the layout along the vertical direction isemphasized if this high frequency component is a predetermined value orgreater. In this case of only changing over from a Bayer array to anarrangement in which the layout along the vertical direction isemphasized, it would also be acceptable to arrange to change over tosuch an arrangement in which the layout along the vertical direction isemphasized, if the high frequency component in the vertical direction isgreater than the high frequency component in the horizontal direction.

It would also be acceptable to arrange to change over the colorarrangement of the pixels from an arrangement in which the layout alongthe horizontal direction is emphasized to an arrangement in which thelayout along the vertical direction is emphasized. In this case, forexample, the changeover from the arrangement in which the layout alongthe horizontal direction is emphasized to the arrangement in which thelayout along the vertical direction is emphasized may be performed ifthe high frequency component in the vertical direction is at least apredetermined amount or more greater than the high frequency componentin the horizontal direction. It should be understood that it would alsobe acceptable to arrange to change over to such an arrangement in whichthe layout along the vertical direction is emphasized, if the highfrequency component in the vertical direction is greater than the highfrequency component in the horizontal direction.

Moreover, it would also be acceptable to arrange to change over thecolor arrangement of the pixels from an arrangement in which the layoutalong the vertical direction is emphasized to an arrangement in whichthe layout along the horizontal direction is emphasized. In this case,for example, the changeover from the arrangement in which the layoutalong the vertical direction is emphasized to the arrangement in whichthe layout along the horizontal direction is emphasized may be performedif the high frequency component in the horizontal direction is at leasta predetermined amount or more greater than the high frequency componentin the vertical direction. It should be understood that it would also beacceptable to arrange to change over to such an arrangement in which thelayout along the horizontal direction is emphasized, if the highfrequency component in the horizontal direction is greater than the highfrequency component in the vertical direction.

Variant Embodiment 6

In the embodiments described above, examples were explained in which thefrequency characteristic detection unit 213 detected the spatialfrequency characteristics in response to half press actuation of therelease button. However, it would also be acceptable to arrange for thefrequency characteristic detection unit 213 to detect the spatialfrequency characteristics in response to full press actuation of therelease button.

Variant Embodiment 7

In the embodiments described above, examples have been explained inwhich the frequency characteristic detection unit 213 detects thespatial frequency characteristic of the subject image as being thesecond spatial frequency characteristic when the high frequencycomponent of the spatial frequency in the vertical direction is greaterthan the high frequency component of the special frequency in thehorizontal direction by the predetermined amount T1 or greater. However,it would also be acceptable to arrange for the frequency characteristicdetection unit 213 to detect the spatial frequency characteristic of thesubject image as being the second spatial frequency characteristic whenthe high frequency component of the spatial frequency in the verticaldirection is greater than the high frequency component of the specialfrequency in the horizontal direction by a predetermined amount T2 orgreater, with this predetermined amount T2 being different from thepredetermined amount T1.

Variant Embodiment 8

In the embodiments described above, examples have been explained inwhich still image photography was performed by employing the flow chartof FIG. 7. However, it would also be possible to apply this imagingdevice in the case of performing photography of a moving image. In thiscase since it is difficult, during moving image photography, to detectthe high frequency components of the spatial frequencies in thehorizontal direction and in the vertical direction with an image sensorhaving an arrangement in which the layout along the horizontal directionis emphasized or an arrangement in which the layout along the verticaldirection is emphasized, accordingly it is preferred to performdetection of the spatial frequency components with a sensor that isdifferent from the image sensor 4, as in the case of the third variantembodiment described above.

Variant Embodiment 9

In the embodiments described above, examples have been explained inwhich the focus detection unit 212 performed processing required forcontrast AF on the basis of the signal from the image sensor 4. However,it would also be acceptable for the focus detection unit 212 to be afocus detection unit that performs processing required for phasedifference AF on the basis of pixel signals from pixels for focusdetection that are provided to the image sensor 4. In this case, in stepS120 described above, the focus detection unit 212 would perform theprocessing required for phase difference AF on the basis of the pixelsignals from the pixels for focus detection while the filter controlunit 214 is controlling the arrangement of the colors of the pixels tobe a Bayer array. Yet further, it would also be acceptable to provide asensor for focus detection that is different from the image sensor 4upon an optical path that is branched off from the optical path of theimaging optical system 31, and to arrange to perform the processingrequired for phase difference AF with this sensor for focus detection.

Variant Embodiment 10

In the embodiments described above, examples have been explained inwhich each of the variable wavelength filters 72 was composed of threefilters, i.e. of an EC layer generating R (red) color, an EC layergenerating G (green) color, and an EC layer generating B (blue) color.However, it would also be acceptable to arrange for each of thewavelength filters 72 to be composed of three filters, these being an EClayer generating Mg (magenta) color, an EC layer generating Ye (yellow)color, and an EC layer generating Cy (cyan) color. In this, by combiningthe transmission wavelengths of these three EC layers, it would becomepossible for the wavelength filters 72 principally to transmit light inany of the Mg, Ye, Cy, W, BK, R, G, or B wavelength regions. Moreover,it would also be acceptable to employ wavelength filters using liquidcrystals for the variable wavelength filters 72.

Variant Embodiment 11

In the embodiments described above, examples have been explained inwhich photodiodes were employed as the photoelectric conversion units.However, it would also be acceptable to arrange to employ aphotoelectric conversion layer as the photoelectric conversion unit.

Variant Embodiment 12

In the embodiments described above, examples have been explained inwhich the image sensor 4 was built as a back side illuminated type.However, it would also be acceptable to build the image sensor 4 as afront side illuminated type, in which the wiring layer 110 is providedon the incident side upon which light is incident.

Variant Embodiment 13

Any of the image sensors and the imaging devices explained in connectionwith the embodiments and variant embodiments described above could beapplied to a camera, a smart phone, a tablet, a camera internallyprovided to a PC, a camera on a vehicle, or a camera mounted to anunmanned aircraft (a drone, a radio-controlled craft, or the like), andso on.

While various embodiments and variant embodiments have been explained inthe above description, the present invention is not to be considered asbeing limited to the details thereof. Other variations that areconsidered to come within the range of the technical concept of thepresent invention are also included within the scope of the presentinvention.

The content of the disclosure of the following application, upon whichpriority is claimed, is hereby installed herein by reference.

Japanese Patent Application 60,527 of 2017 (filed on 27 Mar. 2017).

REFERENCE SIGNS LIST

-   1: imaging device-   3: image sensor-   21: body control unit-   22: analysis unit-   31: imaging optical system-   72: variable wavelength filter-   75: photoelectric conversion unit-   213: frequency characteristic detection unit-   214: filter control unit-   215: first image data generation unit-   216: second image data generation unit

The invention claimed is:
 1. An imaging device, comprising: filters thatare capable of changing between a first state of passing light of afirst wavelength, and a second state of passing light of a secondwavelength, and are arranged in a first direction and a second directionthat is different from the first direction; light reception units thatreceive light that has passed through the filters; and a control unitthat controls the filters to be in the first state or to be in thesecond state, wherein the control unit controls gaps between the filtersin the first state in the first direction and gaps between the filtersin the first state in the second direction.
 2. The imaging deviceaccording to claim 1, wherein: the control unit controls the gaps basedupon a detected light from a subject.
 3. The imaging device according toclaim 1, wherein: the light reception units receive the light that haspassed through the filters, and output signals, and the control unitcontrols the gaps based upon an image generated from the signals.
 4. Theimaging device according to claim 3, further comprising a detection unitthat detects a first spatial frequency component in the first direction,and a second spatial frequency component in the second direction, of theimage generated from the signals outputted from the light receptionunits, wherein: the control unit controls the gaps based upon the firstand second spatial frequency components that have been detected.
 5. Theimaging device according to claim 4, wherein: among spatial frequencycomponents of the image, the first and second spatial frequencycomponents are high frequency components.
 6. The imaging deviceaccording to claim 4, wherein: the control unit generates a first filtergroup made up from a plurality of the filters in the first state, and asecond filter group made up from a plurality of the filters in thesecond state; and the control unit sets gaps between the filters in thefirst filter group in the first direction to be smaller than gapsbetween the filters in the first filter group in the second directionwhen the first spatial frequency component is greater than the secondspatial frequency component, and sets the gaps between the filters inthe first filter group in the second direction to be smaller than thegaps between the filters in the first filter group in the firstdirection when the second spatial frequency component is greater thanthe first spatial frequency component.
 7. The imaging device accordingto claim 6, wherein: the filters have a third state of passing light ofa third wavelength; the control unit generates a third filter group madeup from a plurality of the filters in the third state; the control unitcontrols the filters to be in one of the first state, the second state,and the third state; and if the first spatial frequency component isgreater than the second spatial frequency component, the control unitsets gaps between the second filter group and gaps between the thirdfilter group in the first direction to be respectively smaller than gapsbetween the second filter group and gaps between the third filter groupin the second direction; and, if the second spatial frequency componentis greater than the first spatial frequency component, the control unitsets the gaps between the second filter group and the gaps between thethird filter group in the second direction to be respectively smallerthan the gaps between the second filter group and the gaps between thethird filter group in the first direction.
 8. The imaging deviceaccording to claim 7, wherein: the control unit has a first control ofsetting the gaps between the filters in the first filter group to be thesame, setting the gaps between the filters in the second filter group tobe the same, and setting the gaps between the filters in the thirdfilter group to be the same, in the first direction and in the seconddirection; the control unit has a second control of setting the gapsbetween the first filter group, the gaps between the second filtergroup, and the gaps between the third filter group in the firstdirection, to be respectively smaller than the gaps between the firstfilter group, the gaps between the second filter group, and the gapsbetween the third filter group in the second direction; and the controlunit has a third control of setting the gaps between the first filtergroup, the gaps between the second filter group, and the gaps betweenthe third filter group in the second direction, to be respectivelysmaller than the gaps between the first filter group, the gaps betweenthe second filter group, and the gaps between the third filter group inthe first direction.
 9. The imaging device according to claim 8,wherein: the control unit controls the filters according to the firstcontrol when the difference between the first spatial frequencycomponent and the second spatial frequency component is smaller than acertain amount; the control unit controls the filters according to thesecond control when the first spatial frequency component is greaterthan the second spatial frequency component by a predetermined amount ormore; and the control unit controls the filters according to the thirdcontrol when the second spatial frequency component is greater than thefirst spatial frequency component by a predetermined amount or more. 10.The imaging device according to claim 4, wherein: if the first spatialfrequency component is greater than the second spatial frequencycomponent, the control unit sets the gaps between the filters in thefirst state in the first direction to be smaller than the gaps betweenthe filters in the first state in the second direction.
 11. The imagingdevice according to claim 10, wherein: if the second spatial frequencycomponent is greater than the first spatial frequency component, thecontrol unit sets the gaps between the filters in the first state in thesecond direction to be smaller than the gaps between the filters in thefirst state in the first direction.
 12. The imaging device according toclaim 11, wherein: if the first spatial frequency component is greaterthan the second spatial frequency component, the control unit sets gapsbetween the filters in the second state in the first direction to besmaller than gaps between the filters in the second state in the seconddirection; and, if the second spatial frequency component is greaterthan the first spatial frequency component, the control unit sets thegaps between the filters in the second state in the second direction tobe smaller than the gaps between the filters in the second state in thefirst direction.
 13. The imaging device according to claim 12, wherein:the filters have a third state of passing light of a third wavelength;the control unit controls the filters to be in one of the first state,the second state, and the third state; and if the first spatialfrequency component is greater than the second spatial frequencycomponent, the control unit sets gaps between the filters in the thirdstate in the first direction to be smaller than gaps between the filtersin the third state in the second direction; and, if the second spatialfrequency component is greater than the first spatial frequencycomponent, the control unit sets the gaps between the filters in thethird state in the second direction to be smaller than the gaps betweenthe filters in the third state in the first direction.
 14. The imagingdevice according to claim 13, wherein: the control unit has a firstcontrol of setting the gaps between the filters in the first state to bethe same, setting the gaps between the filters in the second state to bethe same, and setting the gaps between the filters in the third state tobe the same, in the first direction and in the second direction; thecontrol unit has a second control of setting the gaps between thefilters in the first state, the gaps between the filters in the secondstate, and the gaps between the filters in the third state in the firstdirection, to be respectively smaller than the gaps between the filtersin the first state, the gaps between the filters in the second state,and the gaps between the filters in the third state in the seconddirection; and the control unit has a third control of setting the gapsbetween the filters in the first state, the gaps between the filters inthe second state, and the gaps between the filters in the third state inthe second direction, to be respectively smaller than the gaps betweenthe filters in the first state, the gaps between the filters in thesecond state, and the gaps between the filters in the third state in thefirst direction.
 15. The imaging device according to claim 14, wherein:the control unit controls the filters according to the first controlwhen the difference between the first spatial frequency component andthe second spatial frequency component is smaller than a certain amount;the control unit controls the filters according to the second controlwhen the first spatial frequency component is greater than the secondspatial frequency component by a predetermined amount or more; and thecontrol unit controls the filters according to the third control whenthe second spatial frequency component is greater than the first spatialfrequency component by a predetermined amount or more.
 16. An imagesensor, comprising: filters that are capable of changing between a firststate of passing light of a first wavelength, and a second state ofpassing light of a second wavelength, and are arranged in a firstdirection and a second direction that is different from the firstdirection; light reception units that receive light that has passedthrough the filters, and that output signals; and a control unit thatcontrols the filters to the first state or to the second state, basedupon the signals outputted from the light reception units, wherein thecontrol unit controls gaps between the filters in the first state in thefirst direction and gaps between the filters in the first state in thesecond direction.
 17. The image sensor according to claim 16, wherein:the control unit controls the gaps based upon the signals outputted fromthe light reception units.